{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "W6_Homework.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X2kPrtVyXtaP"
      },
      "source": [
        "# CIS-522 Week 6 Homework\r\n",
        "\r\n",
        "__Instructor__: Konrad Kording\r\n",
        "\r\n",
        "__Content creators:__ Hmrishav Bandyopadhyay, Rahul Shekhar, Tejas Srivastava\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2pty3-UGXrIu"
      },
      "source": [
        "In this homework, you will implement image classification problem on a much more complex Dataset, the **CIFAR 10 Dataset**. We will train two models a fully connected network and a Convolutional Neural Network and observe the differences in their performance on images. Further, we will also try to visualize the features learn by some of our Convolutional layers in CNN and answer some questions about ethics and our pod members!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R_VqTwG09NEn",
        "cellView": "form"
      },
      "source": [
        "#@markdown What is your Pennkey and pod? (text, not numbers, e.g. bfranklin)\n",
        "my_pennkey = '' #@param {type:\"string\"}\n",
        "my_pod = 'Select' #@param ['Select', 'euclidean-wombat', 'sublime-newt', 'buoyant-unicorn', 'lackadaisical-manatee','indelible-stingray','superfluous-lyrebird','discreet-reindeer','quizzical-goldfish','ubiquitous-cheetah','nonchalant-crocodile','fashionable-lemur','spiffy-eagle','electric-emu','quotidian-lion','astute-jellyfish', 'quantum-herring']\n",
        "\n",
        "# start timing\n",
        "import time\n",
        "try:t0;\n",
        "except NameError: t0 = time.time()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MPBNHacqXkrC",
        "cellView": "form"
      },
      "source": [
        "# @title Run this cell for imports.\n",
        "\n",
        "# imports\n",
        "import random\n",
        "import pathlib\n",
        "\n",
        "import torch\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import torchvision\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import torch.nn.functional as F\n",
        "import torchvision.transforms as transforms\n",
        "from torchvision.datasets import ImageFolder\n",
        "import torchvision.datasets as datasets\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from torchvision.utils import make_grid\n",
        "from IPython.display import HTML, display\n",
        "\n",
        "from tqdm.notebook import tqdm, trange\n",
        "from time import sleep"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1mWvVdyBWaQ2",
        "cellView": "form"
      },
      "source": [
        "# @markdown Figure Settings\r\n",
        "%config InlineBackend.figure_format = 'retina'\r\n",
        "%matplotlib inline \r\n",
        "plt.style.use(\"https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle\")\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cS8R1OGeoydf",
        "cellView": "form"
      },
      "source": [
        "#@markdown Device Settings<br> It should print 'cuda', if not change your runtime to GPU.\r\n",
        "device =  torch.device('cuda' if torch.cuda.is_available() else 'cpu')\r\n",
        "print(device)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0qzHTBpz9hy4"
      },
      "source": [
        "# Part 0. Exploring the Dataset and Basic Functions\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XqUPARL31PZM"
      },
      "source": [
        "The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The classes in the dataset are:<br>\r\n",
        "['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IYU7E22k0KAj"
      },
      "source": [
        "Let us download the dataset, and build our datatloaders. Set the **BATCH_SIZE** for the dataloaders yourself. \r\n",
        "\r\n",
        "The three dataloaders are accessible by the variables *trainloader*, *validationloader* and *testloader*. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fxh6d3e-l0xo"
      },
      "source": [
        "transform = transforms.Compose(\r\n",
        "    [transforms.ToTensor(),\r\n",
        "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\r\n",
        "\r\n",
        "\r\n",
        "# TODO: set Batch size for dataloaders\r\n",
        "BATCH_SIZE = ...\r\n",
        "\r\n",
        "trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)\r\n",
        "trainloader = torch.utils.data.DataLoader(trainset, batch_size=BATCH_SIZE, shuffle=True, num_workers=2)\r\n",
        "\r\n",
        "\r\n",
        "testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)\r\n",
        "\r\n",
        "# splitting the testset into testset and validationset, with a 80:20 ratio\r\n",
        "validationset, testset = torch.utils.data.random_split(testset, [8000, 2000])\r\n",
        "\r\n",
        "validationloader = torch.utils.data.DataLoader(validationset, batch_size=BATCH_SIZE, shuffle=False, num_workers=2)\r\n",
        "testloader = torch.utils.data.DataLoader(testset, batch_size=BATCH_SIZE, shuffle=False, num_workers=2)\r\n",
        "\r\n",
        "classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QDB8YVwV9b6q"
      },
      "source": [
        "Let us check the sizes of our three sets and print some images. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sqnof9BT9bmf"
      },
      "source": [
        "print(len(trainset))\r\n",
        "print(len(validationset))\r\n",
        "print(len(testset))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5rvwTZenBw4C"
      },
      "source": [
        "# function to show an image\r\n",
        "def imshow(img):\r\n",
        "    img = img / 2 + 0.5     # unnormalize\r\n",
        "    npimg = img.numpy()\r\n",
        "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\r\n",
        "    plt.show()\r\n",
        "\r\n",
        "dataiter = iter(trainloader)\r\n",
        "images, labels = dataiter.next()\r\n",
        "print(images.shape)\r\n",
        "\r\n",
        "imshow(torchvision.utils.make_grid(images[0:4]))\r\n",
        "print('    \\t\\t'.join('%5s' % classes[labels[j]] for j in range(4)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VbXSt-fJg4Dy"
      },
      "source": [
        "Now let us write a train function which basically trains any network by running epochs and also evaluates the results on our validation set in every epoch. We will keep tunable hyperparameters as the parameters for the function, so that we can directly tweak them and call the function. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ax6tvOYamPHy"
      },
      "source": [
        "def train(model, device, train_loader, validation_loader, epochs, criterion, optimizer):\r\n",
        "  # Params:\r\n",
        "  #   model-> instance of your model which extends nn.Module class\r\n",
        "  #   device -> it is set to cuda already and is already stored in a variable called *device*\r\n",
        "  #   train_loader -> dataloader for train dataset, already stored in a variable called *trainloader*\r\n",
        "  #   validation_loader -> dataloader for validation dataset, already stored in a variable called *validationloader*\r\n",
        "  #   epochs -> number of epochs for training, you are expected to experiment and play around with this\r\n",
        "  #   criterion -> the loss function which you decide to use\r\n",
        "  #   optimizer -> optimizer for your gradient descent, do not forget to set an appropriate learning rate\r\n",
        "  #  Returns:  train_loss, train_accuracy, validation_loss, validation_accuracy, all of these are lists storing the values \r\n",
        "\r\n",
        "    train_loss, validation_loss = [], []\r\n",
        "    train_acc, validation_acc = [], []\r\n",
        "    for epoch in range(epochs):\r\n",
        "        model.train()\r\n",
        "        running_loss = 0.\r\n",
        "        correct, total = 0, 0 \r\n",
        "        # TODO: train on the train dataset\r\n",
        "        with tqdm(train_loader, unit='batch') as tepoch:\r\n",
        "            tepoch.set_description('Training: ')\r\n",
        "            ...\r\n",
        "            \r\n",
        "            \r\n",
        "\r\n",
        "        \r\n",
        "        \r\n",
        "                \r\n",
        "        # TODO: evaluate on validation data\r\n",
        "        model.eval()\r\n",
        "        running_loss = 0.\r\n",
        "        correct, total = 0, 0 \r\n",
        "        with tqdm(validation_loader, unit='batch') as tepoch:\r\n",
        "            tepoch.set_description('Validation: ')\r\n",
        "            ...\r\n",
        "         \r\n",
        "    \r\n",
        "    return train_loss, train_acc, validation_loss, validation_acc "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NUkhShTOm00R"
      },
      "source": [
        "Now, let us write a test function, which returns the accuracy of a trained model on the testdataset, given the testloader. \r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UJC7BLkSm2_2"
      },
      "source": [
        "def test(model, device, data_loader):\r\n",
        "  # Params:\r\n",
        "  #   model-> instance of your trained model which extends nn.Module class\r\n",
        "  #   device -> it is set to cuda already and is already stored in a variable called *device*\r\n",
        "  #   data_loader -> dataloader for test dataset, already stored in a variable called *testloader*\r\n",
        "  # Returns:\r\n",
        "  #   accuracy(float)-> percentage of correct predictions made by your trained model\r\n",
        "    model.eval()\r\n",
        "    correct = 0\r\n",
        "    total = 0\r\n",
        "    for data in data_loader:\r\n",
        "      # TODO: fill in the test function\r\n",
        "      ...\r\n",
        "\r\n",
        "\r\n",
        "\r\n",
        "    return acc"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8uvt-pw0pqxn"
      },
      "source": [
        "Below, we have provided a function to plot your train loss, test loss, train accuracy and test accuracy, during the training process. They take in arguments, which were the four lists returned by your train function. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CeCIzBG1ppkc"
      },
      "source": [
        "# code to plot loss and accuracy\r\n",
        "def plot_loss_accuracy(train_loss, train_acc, validation_loss, validation_acc):\r\n",
        "    epochs = len(train_loss)\r\n",
        "    fig, (ax1, ax2) = plt.subplots(1, 2)\r\n",
        "    ax1.plot(list(range(epochs)), train_loss, label='Training Loss')\r\n",
        "    ax1.plot(list(range(epochs)), validation_loss, label='Validation Loss')\r\n",
        "    ax1.set_xlabel('Epochs')\r\n",
        "    ax1.set_ylabel('Loss')\r\n",
        "    ax1.set_title('Epoch vs Loss')\r\n",
        "    ax1.legend()\r\n",
        "\r\n",
        "    ax2.plot(list(range(epochs)), train_acc, label='Training Accuracy')\r\n",
        "    ax2.plot(list(range(epochs)), validation_acc, label='Validation Accuracy')\r\n",
        "    ax2.set_xlabel('Epochs')\r\n",
        "    ax2.set_ylabel('Accuracy')\r\n",
        "    ax2.set_title('Epoch vs Accuracy')\r\n",
        "    ax2.legend()\r\n",
        "    fig.set_size_inches(15.5, 5.5)\r\n",
        "    plt.show()\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3qTosRRlnu1q"
      },
      "source": [
        "# Part 1: FeedForward Neural Network\r\n",
        "\r\n",
        "In this section, we will implement a Feed forward Neural Network that can classify the images in the CIFAR 10 dataset. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M9QkFYgn1U2V"
      },
      "source": [
        "Here, we define our FFN class which will consists of all the layers and the forward function. Feel free to play around by introducing different types of regularization, architectures, learning rates, epochs, and batch sizes, in order to achieve maximum accuracy, except the constraint that do not use convolutional layers in the model. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bVF8XNgSIi3t"
      },
      "source": [
        "## Architecture"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2Re6fjrqmNxO"
      },
      "source": [
        "class FFN(nn.Module):\r\n",
        "  def __init__(self):       \r\n",
        "    super(FFN, self).__init__()\r\n",
        "    # TODO: fill in the layers\r\n",
        "    self.linear = ...\r\n",
        "\r\n",
        "  # TODO: define the forward pass\r\n",
        "  def forward(self, x):\r\n",
        "\r\n",
        "    return x\r\n",
        "\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TSbgt3aZInJm"
      },
      "source": [
        "## Training"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h0tjSGwx2K-3"
      },
      "source": [
        "Now, let us intantiate our FFN class, define the loss, the optimizer and call the train function. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f21cwe-dr9QX"
      },
      "source": [
        "# TODO: set criterion, optimizer and other hyperparameters and train\r\n",
        "net1 = FFN().to(device)\r\n",
        "criterion = ...\r\n",
        "optimizer = ...\r\n",
        "num_epochs = ...\r\n",
        "train_loss, train_acc, validation_loss, validation_acc = train(net1, device, trainloader, validationloader, num_epochs, criterion, optimizer)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nZuZ8tmtIqIq"
      },
      "source": [
        "## Training Plots"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n5k3Ng4X3Gc2"
      },
      "source": [
        "Now, let us plot the train and validation curves. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vXxKn6tJsYEK"
      },
      "source": [
        "plot_loss_accuracy(train_loss, train_acc, validation_loss, validation_acc)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nZUNCjGAFVoX"
      },
      "source": [
        "Take a screenshot of your Training Curves (Both Accuracy and Loss) and submit it in the AirTable Form at the end of the Worksheet."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-_S9sqmcIurm"
      },
      "source": [
        "## Test Accuracy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jMbbT8jU3MrQ"
      },
      "source": [
        "And test our trained network on the test set. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uV8LUhKOt53H"
      },
      "source": [
        "test(net1, device, testloader)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lKFjXo2_H9aP"
      },
      "source": [
        "And fill up the accuracy value you get. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SbasJZLmGaxR"
      },
      "source": [
        "ffn_accuracy = '''xx.xx %'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CLS6YvCA8_aE"
      },
      "source": [
        "Try to reach a test accuracy of atleast 45% with the Fully Connected Network. Try for even more!\r\n",
        "\r\n",
        "You may try different models and variations and then submit the results (training curves) and the accuracy of your best model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GdSDRIxu_Y56"
      },
      "source": [
        "## Discuss\r\n",
        "\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "5J-Qy5e_3kfb"
      },
      "source": [
        "#@markdown Now, discuss the hyperparameters and architectures you chose. Which model led to the best performance? Why? (200 words)\r\n",
        "import time\r\n",
        "try:t1;\r\n",
        "except NameError: t1 = time.time()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1F1t6sv_q2K"
      },
      "source": [
        "ffn_discussion = ''' \r\n",
        "write your answer here\r\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ls3EKr4846PS"
      },
      "source": [
        "# Part 2: Convolutional Neural Network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BS71XpEt8yDA"
      },
      "source": [
        "Now, design a Convolutional Neural Network, for the same problem. Again, try to play around with various architectures, regularizations, activations, learning rates, optmizers and try to leverage the power of CNNs!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "olHo2tLEDAoe"
      },
      "source": [
        "**Note:** Put all your Convolutional layers, activations in the first Sequential module, instead of defninig different layers and making an explicit forward pass.\r\n",
        "Similarly, add all the subsequent fully connected layers in the second Sequential module, which comes after flattening the output from the convolutional layers.\r\n",
        "\r\n",
        "**Do not change the forward pass code here.** \r\n",
        "<br>\r\n",
        "This will help you run the next section on Visualization on your chosen architecture and visualize the feature maps of different convolutional layers. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F8kIqEMUI5le"
      },
      "source": [
        "## Architecture"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YVtNorDY71Lm"
      },
      "source": [
        "\r\n",
        "class CNN(nn.Module):\r\n",
        "    def __init__(self):\r\n",
        "        super().__init__()\r\n",
        "        self.features = nn.Sequential(\r\n",
        "            # TODO: fill the convolutional layers\r\n",
        "            nn.Conv2d(...),\r\n",
        "            nn.ReLU(),\r\n",
        "            ...\r\n",
        "        )\r\n",
        "\r\n",
        "        self.fc = nn.Sequential(\r\n",
        "            # TODO: fill the fully connected layers that follow the conv module\r\n",
        "            nn.Linear(...),\r\n",
        "            \r\n",
        "         )\r\n",
        "        \r\n",
        "    # DO NOT CHANGE THE FORWARD PASS\r\n",
        "    def forward(self, x):\r\n",
        "        x = self.features(x)\r\n",
        "        x = torch.flatten(x, 1)\r\n",
        "        output=self.fc(x)\r\n",
        "        return output"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4d2BS0XI7zV"
      },
      "source": [
        "## Training"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ty9Oygt8Enwe"
      },
      "source": [
        "Calling the train function for the ConvNet."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Cz4x38y71Lo"
      },
      "source": [
        "# TODO: fill the hyperparameters and train the model\r\n",
        "net2 = CNN().to(device)\r\n",
        "criterion = ...\r\n",
        "optimizer = ...\r\n",
        "num_epochs = ...\r\n",
        "train_loss, train_acc, validation_loss, validation_acc = train(net2, device, trainloader, validationloader, num_epochs, criterion, optimizer)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ynl60dMXI-10"
      },
      "source": [
        "## Training Plots"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t__-RhyhE7C1"
      },
      "source": [
        "Now plot your training curves!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X_uJupIo71Lp"
      },
      "source": [
        "plot_loss_accuracy(train_loss, train_acc, validation_loss, validation_acc)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wnYNazivJLhd"
      },
      "source": [
        "## Test Accuracy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qhMLzCl0E_DY"
      },
      "source": [
        "And, test your trained model on the Test Dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X9ergUuc71Lq"
      },
      "source": [
        "test(net2, device, testloader)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "88BR57f7IRMO"
      },
      "source": [
        "And, fill up the accuracy here."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yLu7fsQeHvly"
      },
      "source": [
        "cnn_accuracy = '''xx.xx %'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X_8JrjqgFE9S"
      },
      "source": [
        "Try to reach an accuracy of atleast 52%.Of course try to get your test accuracy even higher, and leverage the true powers of CNNs!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EXz_yDqvJT7c"
      },
      "source": [
        "## Discussion"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "aGOLsp1d31Yd"
      },
      "source": [
        "#@markdown Discuss the hyperparameters and architectures you chose. Which model led to the best performance? Did your CNN outperform your FFN? Why? (200 words)\r\n",
        "import time\r\n",
        "try:t2;\r\n",
        "except NameError: t2 = time.time()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3mx3pslXHukJ"
      },
      "source": [
        "cnn_discussion = '''\r\n",
        "Write your response here\r\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "739NtnJJJXKL"
      },
      "source": [
        "# Part 3: Visualizations: What do the features look like?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bwFOE6eeYP_e",
        "cellView": "form"
      },
      "source": [
        "dataiter = iter(trainloader)\n",
        "images, labels = dataiter.next()\n",
        "input_img=images[random.randrange(0,BATCH_SIZE,1)].unsqueeze(0)\n",
        "img_original=input_img.squeeze().permute(1,2,0).numpy()\n",
        "\n",
        "\n",
        "Layer_Name= 3 #@param {type:\"slider\", min:1, max:54, step:1}\n",
        "Layer_Name=str(Layer_Name)\n",
        "Num_Filters = 23 #@param {type:\"slider\", min:1, max:64, step:1}\n",
        "from skimage.transform import resize\n",
        "import matplotlib.pyplot as plt\n",
        "def temp_rem(model,key,input_img):\n",
        "    model.eval()\n",
        "    flag=0\n",
        "    alt_=[]    \n",
        "    for search in model._modules['features']._modules.keys():\n",
        "        if flag==1:\n",
        "            break\n",
        "        if key==search:\n",
        "            flag=1\n",
        "        alt_.append(model._modules['features']._modules[search])\n",
        "    return torch.nn.Sequential(*alt_).to(device)(input_img.to(device)).squeeze().detach().cpu().numpy()\n",
        "\n",
        "def plot(data,Num_Filters,img_original):\n",
        "    import matplotlib.pyplot as plt\n",
        "    i=0\n",
        "    flag=0\n",
        "    #print(len(data))\n",
        "    if Num_Filters>len(data):\n",
        "        print(\"Please set Num_Filters parameter less than or equal to the number of filters present in the layer ({} in this case)\".format(len(data)))\n",
        "        return\n",
        "    if Num_Filters<8:\n",
        "        fig, ax = plt.subplots(1,Num_Filters)\n",
        "        for col in ax:\n",
        "            if flag==0:\n",
        "                col.imshow(resize(img_original,(256,256)))    \n",
        "                col.set_xticklabels([])\n",
        "                col.set_yticklabels([])\n",
        "                \n",
        "                flag=1\n",
        "            else:\n",
        "                col.imshow(resize(data[i],(256,256)))\n",
        "                col.set_xticklabels([])\n",
        "                col.set_yticklabels([])\n",
        "                \n",
        "            i=i+1\n",
        "        \n",
        "        plt.tight_layout()\n",
        "        plt.subplots_adjust(wspace=0, hspace=0)\n",
        "        plt.show()\n",
        "        return\n",
        "\n",
        "    i=0    \n",
        "    if Num_Filters<16:\n",
        "        ncols=4\n",
        "    else:\n",
        "        ncols=8\n",
        "\n",
        "    nrows=Num_Filters//ncols\n",
        "    fig, ax = plt.subplots(nrows=nrows, ncols=ncols)\n",
        "    for rows in ax:\n",
        "        for cols in rows:\n",
        "            if flag==0:\n",
        "                cols.imshow(resize(img_original,(256,256)))    \n",
        "                cols.set_xticklabels([])\n",
        "                cols.set_yticklabels([])\n",
        "                flag=1\n",
        "            else:\n",
        "                cols.imshow(resize(data[i],(256,256)))\n",
        "                cols.set_xticklabels([])\n",
        "                cols.set_yticklabels([])\n",
        "                \n",
        "            i=i+1\n",
        "        \n",
        "        \n",
        "    fig.set_size_inches(ncols*2.5,nrows*2.5)\n",
        "    plt.tight_layout()\n",
        "    plt.subplots_adjust(wspace=0, hspace=0)\n",
        "    plt.show()\n",
        "    \n",
        "\n",
        "model = net2\n",
        "plot(temp_rem(model,Layer_Name,input_img),Num_Filters,img_original)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n1MSQY72znLY"
      },
      "source": [
        "## Discussion"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FYTTTeHTzk4Z"
      },
      "source": [
        " Describe some of the convolutional feature maps for certain images. What do you think they are representing? Do deeper convolutions result in different feature representations? (200 words)\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "KloaHuRU39ub"
      },
      "source": [
        "#@markdown Describe some of the convolutional feature maps for certain images. What do you think they are representing? Do deeper convolutions result in different feature representations? (200 words)\r\n",
        "import time\r\n",
        "try:t3;\r\n",
        "except NameError: t3 = time.time()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xNkJJXyw2EJj"
      },
      "source": [
        "visualization_discuss = '''\r\n",
        "Write your response here\r\n",
        "\r\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U-el6vZazvsX"
      },
      "source": [
        "# Part 4: Deep Learning, Deep Thinking\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EKHImQaKvrWs"
      },
      "source": [
        "## Part 4.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Ww0XUthvp1J"
      },
      "source": [
        "Okay, so you all just solved a classification task using the CIFAR Dataset. Before we move ahead, please read the following to articles. \r\n",
        " https://excavating.ai/ and https://medium.com/analytics-vidhya/an-introduction-to-fairness-in-machine-learning-62ef827e0020 (explains the fundamentals of measuring fairness in machine learning models).\r\n",
        " \r\n",
        " \r\n",
        "<br><br> \r\n",
        "What do these readings suggest? There may be problems with unbalanced data and biases present in data. Now, go back and reflect on what you just did. Did you consider these problems when you designed your convnet? Write your thoughts (about 100 words)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a8WrSAEHv6V-"
      },
      "source": [
        "bias_response = '''\r\n",
        "Your response here\r\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SAgCpqL0wFr0"
      },
      "source": [
        "Now, its time to see what those implications are on the models you just trained. <br>\r\n",
        "\r\n",
        "<br>\r\n",
        "\r\n",
        "\r\n",
        "\r\n",
        "> Report the results of the independence and separation criteria you just read about in the medium article i.e report the accuracy for each class (independence) and report the false positive and false negative for each class (separation). <br>You can write separate code for implementing this. \r\n",
        "<br>\r\n",
        "Submit a screenshot of your results in the AirTable Form at the end. (Do not worry about the format, it should be interpretable and readable)\r\n",
        "\r\n",
        "\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IqaANK34Qsb2"
      },
      "source": [
        "# Your code here\r\n",
        "\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pJwRMHDmwalr"
      },
      "source": [
        "## Part 4.2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeD6--thwYue"
      },
      "source": [
        "Let’s take a step back and consider the following questions \r\n",
        "\r\n",
        "1.   why might data be biased or imbalanced in the first place? Who gets to determine the labels of these data, and what might be some ethical concerns here?\r\n",
        "2.   what are ethical implications of not investigating the data beforehand? Give a concrete example of some strategies to mitigate this.\r\n",
        "<br><br>\r\n",
        "Post your thoughts on your pods slacks channel and have a discussion amongst your pod members."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iWFKVVaLz2bs"
      },
      "source": [
        "# Part 5: Better Know a Pod"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QUZQBnO4z_xM"
      },
      "source": [
        "Discuss with two other members of your pod. What is something they've always wanted to learn but have never gotten the chance to? (~100 words each)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "kY3cvc-N4IbK"
      },
      "source": [
        "#@markdown Discuss with two other members of your pod. What is something they've always wanted to learn but have never gotten the chance to? (~100 words each)\r\n",
        "import time\r\n",
        "try:t4;\r\n",
        "except NameError: t4 = time.time()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jEcY5P7S0CeI"
      },
      "source": [
        "know_a_pod_1 = '''\r\n",
        "  First response here\r\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FAYb3dTP0IMC"
      },
      "source": [
        "know_a_pod_2 = '''\r\n",
        "  Second Response here\r\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uyHy54ZxPjPt"
      },
      "source": [
        "# Submit your responses\r\n",
        "Please run the following cell and then press \"Submit\" so we can record your responses."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JAiufZ39Lpq8",
        "cellView": "form"
      },
      "source": [
        "\r\n",
        "#@markdown #Run Cell to Show Airtable Form\r\n",
        "#@markdown ##**Confirm your answers, upload your screenshots, paste the notebook link and then click \"Submit\"**\r\n",
        "\r\n",
        "import time\r\n",
        "import numpy as np\r\n",
        "import urllib.parse\r\n",
        "from IPython.display import IFrame\r\n",
        "def prefill_form(src, fields: dict):\r\n",
        "  '''\r\n",
        "  src: the original src url to embed the form\r\n",
        "  fields: a dictionary of field:value pairs,\r\n",
        "  e.g. {\"pennkey\": my_pennkey, \"location\": my_location}\r\n",
        "  '''\r\n",
        "  prefill_fields = {}\r\n",
        "  for key in fields:\r\n",
        "      new_key = 'prefill_' + key\r\n",
        "      prefill_fields[new_key] = fields[key]\r\n",
        "  prefills = urllib.parse.urlencode(prefill_fields)\r\n",
        "  src = src + prefills\r\n",
        "  return src\r\n",
        "\r\n",
        "\r\n",
        "#autofill time if it is not present\r\n",
        "try: t0;\r\n",
        "except NameError: t0 = time.time()\r\n",
        "try: t1;\r\n",
        "except NameError: t1 = time.time()\r\n",
        "try: t2;\r\n",
        "except NameError: t2 = time.time()\r\n",
        "try: t3;\r\n",
        "except NameError: t3 = time.time()\r\n",
        "try: t4;\r\n",
        "except NameError: t4 = time.time()\r\n",
        "\r\n",
        "\r\n",
        "#autofill fields if they are not present\r\n",
        "#a missing pennkey and pod will result in an Airtable warning\r\n",
        "#which is easily fixed user-side.\r\n",
        "try: my_pennkey;\r\n",
        "except NameError: my_pennkey = \"\"\r\n",
        "\r\n",
        "try: my_pod;\r\n",
        "except NameError: my_pod = \"Select\"\r\n",
        "\r\n",
        "try: ffn_accuracy;\r\n",
        "except NameError: ffn_accuracy = \"\"\r\n",
        "\r\n",
        "try: ffn_discussion;\r\n",
        "except NameError: ffn_discussion = \"\"\r\n",
        "\r\n",
        "try: cnn_accuracy;\r\n",
        "except NameError: cnn_accuracy = \"\"\r\n",
        "\r\n",
        "try: cnn_discussion;\r\n",
        "except NameError: cnn_discussion = \"\"\r\n",
        "\r\n",
        "try: visualization_discuss;\r\n",
        "except NameError: visualization_discuss = \"\"\r\n",
        "\r\n",
        "try: know_a_pod_1;\r\n",
        "except NameError: know_a_pod_1 = \"\"\r\n",
        "\r\n",
        "try: know_a_pod_2;\r\n",
        "except NameError: know_a_pod_2 = \"\"\r\n",
        "\r\n",
        "try: bias_response;\r\n",
        "except NameError: bias_response = \"\"\r\n",
        "\r\n",
        "times = np.array([t1, t2,t3,t4])-t0\r\n",
        "\r\n",
        "fields = {\"my_pennkey\": my_pennkey,\r\n",
        "          \"my_pod\": my_pod,\r\n",
        "          \"ffn_accuracy\":ffn_accuracy,\r\n",
        "          \"ffn_discussion\": ffn_discussion,\r\n",
        "          \"cnn_accuracy\": cnn_accuracy,\r\n",
        "          \"cnn_discussion\": cnn_discussion,\r\n",
        "          \"visualization_discuss\":visualization_discuss,\r\n",
        "          \"know_a_pod_1\":know_a_pod_1,\r\n",
        "          \"know_a_pod_2\":know_a_pod_2,\r\n",
        "          \"bias_response\":bias_response,\r\n",
        "          \"cumulative_times\": times}\r\n",
        "\r\n",
        "src = \"https://airtable.com/embed/shrp0bcg2kvnLqKDN?\"\r\n",
        "#now instead of the original source url, we do: src = prefill_form(src, fields)\r\n",
        "display(IFrame(src = prefill_form(src, fields), width = 800, height = 400))"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}